Walking, swallowing, respiration and many other key functions in humans
and other animals are controlled by Central
Pattern Generators (CPGs). In essence, CPGs are small, autonomous
neural networks that produce rhythmic outputs, usually found in animal's
spinal cords rather than their brains. Their relative simplicity and
obvious success in biological systems has led to some success
in using CPGs in robotics. However, current systems are restricted
to very simple CPGs (e.g., restricted to a single walking gait). A
recent breakthrough at the BCCN
at the University of Göttingen, Germany has now allowed to achieve
11 basic behavioral patterns (various gaits, orienting, taxis,
self-protection) from a single CPG, closing in on the 10–20 different
basic behavioral patterns found in a typical cockroach. The trick: Work
with a chaotic, rather than a stable periodic CPG regime. For more on
CPGs, listen to the latest episode of the Robots
podcast on Chaos Control, which interviews Poramate
Manoonpong, one of the lead researchers in Göttingen, and Alex Pitti from the University of Tokyo who uses chaos
controllers that can synchronize to the dynamics of the body they are
controlling.
otakuhacks on 29 Sep 20Transformers have now become the defacto standard for NLP tasks. Originally developed for sequence transduction processes such as speech recognition, translation, and text to speech, transformers work by using convolutional neural networks together with attention models, making them much more efficient than previous architectures. And although transformers were developed for NLP, they've also been implemented in the fields of computer vision and music generation. However, for all their wide and varied uses, transformers are still very difficult to understand, which is why I wrote a detailed post describing how they work on a basic level. It covers the encoder and decoder architecture, and the whole dataflow through the different pieces of the neural network. In this post, we'll get deeper into looking at transformers by implementing our own English to German language translator.
